1. Field of the Invention
The present invention relates to a performance evaluating apparatus for calculating a performance value representing the performance of an information processing apparatus for carrying out a processing operation.
2. Description of the Related Art
One example of time-sharing processing is a virtual machine technology for utilizing the central processing unit (CPU) of an information processing apparatus (computer) on a time-sharing basis. FIG. 1 of the accompanying drawings shows a system using a virtual machine by way of example. The virtual machine technology allows a plurality of operating systems (OS) to be time-shared by one CPU.
Each of the operating systems executed by the single CPU 211 is called a virtual machine (VM) 202a, 202b. Specifically, each virtual machine 202a, 202b is assigned the resources of physical machines such as memory 212, a disk 213, a network card (NIC) 214, etc. as well as the CPU 211, and can freely use the assigned resources. Therefore, each virtual machine can execute an OS (guest OS) 204a, 204b and application programs (applications) 203a, 203b as if it were a single physical machine.
The CPU 211 of the computer 201 which is time-shared by the virtual machines 202a, 202b is referred to as a physical CPU, and a virtual CPU that is assigned to each virtual machine is referred to as a virtual CPU. The virtual machine technology, including virtualization software 205, is used in the field of computer systems for better computer utilization and higher security between virtual machines.
A computer system based on virtual machine technology makes it possible to establish an upper limit for resources of a CPU (CPU utilization) that can be used by each virtual machine to thereby achieve independence between a plurality of virtual machines constructed on the physical CPU. For determining an upper limit for resources assigned to each virtual machine, there is a trade-off between the following two requirements:
The first requirement is a requirement for more virtual machines to be constructed on a physical machine to realize cost reduction. The second requirement is a requirement for sufficient resources to be given to each virtual machine to guarantee the performance of applications run on each virtual machine.
In order to determine minimum resources which meet demands for performance against the trade-off, a need arises for a performance evaluating apparatus to calculate a performance value representing the performance depending on resources assigned to each virtual machine.
A known performance evaluating apparatus will be described below.
Document “Kino Issei “Queuing network”, Asakura Publishing Co., Ltd., Nov. 25, 2002, pp. 136 through 142” (hereinafter referred to as Document 1) discloses a performance evaluating apparatus which addresses ordinary computer systems, rather than virtual machines. Document 1 deals with a performance calculating method for an information processing apparatus having a CPU and a disk. However, an upper limit for CPU utilization is not established in the information processing apparatus.
According to the information processing apparatus, a queuing model is generated by regarding each CPU and each disk as a queue, and an average response time is calculated according to the queuing theory based on a CPU execution time (a time in which the CPU performs its processing operation) and a disk execution time (a time in which the disk (storage device) performs a storing process (writing process or reading process)).
The average response time represents the average value of a response time per job when a plurality of jobs are executed (a time consumed after the execution of one job is started until the execution of the job is completed and the execution of the next (other) job is started).
Document “Anatoliy Rikun, Yiping Ding “Optimization with Service Level Objectives in Virtual Environment” Proceedings of Computer Measurement Group (CMG) Conference, 2006” (hereinafter referred to as Document 2) discloses a performance evaluating apparatus for calculating a CPU execution time when an upper limit for CPU utilization is established. If the number of processes which a physical CPU can process per unit time is represented by μ, and an upper limit for CPU utilization which is established with respect to a virtual machine is represented by α (0<α<1), then the performance evaluating apparatus calculates an average response time according to the queuing theory by regarding the number of requests (the number of processes) which the virtual machine can process per unit time as α·μ.
However, the above performance evaluating apparatus suffers a problem in that the calculated response time of the virtual machine tends to be larger than an actual value, i.e., the performance evaluating apparatus tends to estimate the performance as too low. This problem will be described in detail below.
An example in which an application for sequentially processing jobs, i.e., for performing a process, using a CPU and a disk is run on virtual machines will be described below. The average response time of the application is calculated as a performance value of the virtual machines. The sequential process refers to a process for waiting for one job to be finished and then performing the next job.
A system in which a physical CPU is shared by two virtual machines and which has a single disk will be described by way of example below. According to Document 1, the sequential process is expressed as a batch process, and is represented by a queuing model that is indicated by a closed system shown in FIG. 2 of the accompanying drawings.
In FIG. 2, the two virtual machines are expressed as independent queues. It is assumed that upper limits for CPU utilization which are assigned to the respective virtual machines are indicated by α1, α2 (0<α1, 0<α2, α1+α2≦1), respectively.
As described in Document 2, the processing capability of each virtual CPU is considered to be α1 times or α2 times the processing capability of the physical CPU. For calculating an average response time according to the above queuing model, the convolutional method disclosed in Document “To-shio Kameda, Ketsu Ri, Kino Issei “Basics and Applications of Performance Evaluation (Information Mathematics Lecture)”, Kyoritsu Shuppan Co., Ltd., 1998, pp. 134 through 151” can be used.
FIG. 3 of the accompanying drawings shows measured values of the average response time and calculated values (estimated value) thereof which are produced by the above performance evaluating apparatus at the time an application for sequentially processing jobs wherein the average value of a CPU basic execution time per job is 0.134 second and the average value of a disk basic execution time is 0.939 second is run on virtual machine 1.
The CPU basic execution time refers to the execution time of a processing operation performed by the CPU in case where 100% of the processing capability of the physical CPU is utilized, and does not include queuing times according to other processes. The disk basic execution time refers to the execution time of a storing operation performed by the disk in case where 100% of the processing capability of the physical CPU is utilized, and does not include queuing times according to other processes.
If the upper limit for CPU utilization by a virtual machine is 10%, then the virtual machine is handled as a CPU having an ability to perform a 0.746 (=(1/0.134)×(1/10)) process in the queuing model.
To simplify calculations of the queue, it is assumed that the distribution of the execution times for the jobs is represented by an exponential distribution, and the response time is measured while virtual machine 2 does not execute the application.
The graph shown in FIG. 3 indicates how the average response time varies when the upper limit for CPU utilization by the virtual machine is changed. A comparison of the results indicates that if the upper limit for CPU utilization by a virtual machine is 10%, the calculated average response time is about twice the measured average response time.
If the performance value is estimated as too low when the system is designed, then since extra computer resources need to be provided, the cost of the system which is constructed is increased. Conversely, if the performance value is estimated as too high, then the performance value expected when the system is designed cannot be reached when the system is in actual operation.
The reason why the average response time calculated by the performance evaluating apparatus and the measured average response time are different from each other is that the operation of a virtual machine assumed by the performance evaluating apparatus is different form the operation of an actual virtual machine. Differences between the operation of a virtual machine assumed by the performance evaluating apparatus and the operation of an actual virtual machine will be described below.
As shown in FIGS. 4 and 5 of the accompanying drawings, an actual virtual machine uses 100% of the CPU power within an assigned time. If an upper limit α for CPU utilization is established for the virtual machine, then a time in which the virtual machine can perform a process using the CPU within a certain time T is limited to α·T.
In order to keep the upper limit for CPU utilization established for the virtual machine, a time is inserted in which the virtual machine cannot use the CPU. The time inserted for keeping the upper limit for CPU utilization will hereinafter be called idle time (waiting time) tidle. Idle time tidle has duration (1−α)·T.
If there are a plurality of virtual machines, then other virtual machines perform their processes using the CPU in idle time tidle. If a sequential process is performed, then since the CPU is not used while the disk is performing its storing operation, the disk execution time is included in idle time tidle.
If disk execution time tdisk is longer than idle time tidle (see FIG. 4), then CPU utilization is less than the upper limit because the CPU is not used longer than idle time tidle during the storing operation of the disk even though the processing operation of the CPU and the storing operation of the disk are repeatedly carried out. The response time of the process at this time is equal to the sum of the CPU execution time and the disk execution time, and does not depend on the upper limit for CPU utilization, i.e., idle time tidle.
If disk execution time tdisk is shorter than idle time tidle (see FIG. 5), then the storing operation of the disk is completed within idle time tidle. The response time at this time depends on the duration of idle time tidle, and varies in inverse proportion to the upper limit for CPU utilization.
The operation of the CPU assumed by the above performance evaluating apparatus is shown in FIG. 6 of the accompanying drawings.
If the upper limit for CPU utilization established for a virtual machine is represented by α(0<α<1), then the performance evaluating apparatus assumes a CPU whose processing capability (CPU power) is equal to the processing capability (CPU power) of a physical CPU as it is reduced α times, as a virtual CPU. The execution time of a processing operation performed by the CPU is equal to 1/α of the CPU execution time in case 100% of the CPU power is utilized. Therefore, a reduction in the value of α results in an increase in the CPU execution time. The execution time of a storing operation performed by the disk does not vary depending on the CPU power.
Inasmuch as the above performance evaluating apparatus assumes that the processing operation is performed by the CPU even when the processing operation of the CPU is actually completed and the storing operation of the disk is performed, the performance evaluating apparatus estimates the response time as too long.
Furthermore, as described above, parameters that affect the response time vary depending on idle time tidle and disk execution time tdisk (see FIG. 4 and FIG. 5). As a result, the performance evaluating apparatus is problematic in that it is unable to calculate a performance value with high accuracy.